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Status of “Choice of College’ Rule 


As is well known, a candidate registering 
for college-entrance examinations given by 
the College Entrance Examination Board is 
required to list on his application blank his 
order of preference for the colleges to 
which he wishes his test scores reported. 
This requirement has been under study by 
a group of seven school and college repre- 
sentatives appointed as a special committee 
by the Chairman of the Board in April, 
1947. In October, 1947, this committee pre- 
sented two recommendations to the Execu- 
tive Committee of the Board, as follows: 

(Continued on page 34) 


RESEARCH 


EXAMINATION BOARD 


BY J. E. TERRAL AND HAROLD L. CRANE 


Department of Test Administration 


Last year the College Entrance Exami- 
nation Board scored over 200,000 test 
papers from over 60,000 candidates seeking 
admission to college. This is obviously a 
large operation—and it must be performed 
under the mutually conflicting requirements 
of speed, accuracy, and economy. Adding 
just one cent, for example, to the cost of 
scoring each paper would increase the total 
scoring cost by over $2,000. 


DEGREE OF ACCURACY 


The concern of the present article, how- 
ever, is primarily with accuracy. It will not 
be pretended that complete, perfect ac- 
curacy is always attained: no extensive 
operation on earth, whether human or 
mechanical, is completely free from occa- 
sional minor error. According to careful 
check, however, the degree of accuracy 
attained in the scoring of the College Board 
tests is high. Thus, of about 8,000 scores on 
various tests, a careful check showed only 
3 scores in error by two raw-score points, 
and only 3 scores in error by more than two 
raw-score points (the largest error being 
ten points). In the case of the Scholastic 
Aptitude Test (taken by more candidates 

(Continued on page 38) 
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“Choice of College” Rule 


(Continued from page 33) 


Recommendation 1. Each candidate 
should be required to designate not more 
than two colleges as specially preferred 
(first or second choice, without distinc- 
tion); and to designate any other colleges 
of kis choice as not specially preferred 
(again without distinction among the col- 
leges so designated ). 

Recommendation 2. Colleges should be 
urged to indicate, in the Board’s ANNUAL 
Hanpsook, their respective policies with re- 
gard to the. use made of the preference- 
designations in deciding admissions. 

At the October, 1947 meeting of the 
Board, only the second recommendation of 
the special committee’s report was recom- 
mended by the Executive Committee for 
adoption by the Board. Because of the need 
for time to consider the proposed merger of 
educational testing agencies (see page 36 in 
this issue of the Review), the recommen- 
dations both of the special committee and 
the Executive Committee were tabled. The 
question is scheduled for discussion at the 
April, 1948 meetings of the Board and its 
committees. 

For the immediate present, then, the 
candidate is still required to list on his ap- 
plication blank his order of preference for 
the colleges to which he wishes his scores 
reported. 
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DELAR 
PRESIDENT KATHARINE E. McBrRIDE 


Member of the Executive Committee of the College 
Entrance Examination Board, and Member of the 
Board of Trustees of the Educational Testing 
Service 

Both an outstanding educator and a psy- 
chologist who has specialized in mental 
measurements, President McBride has given 
valuable leadership in the Board’s testing 
program. For the past two years she served 
with distinction both as a member of the 
Advisory Committee on Research and De- 
velopment and as chairman of the Commit- 
tee on Examinations; in October 1947 she 
was elected a member of the Executive 
Committee. She has devoted herself gener- 
ously to the problems of the Board. 

As a member of the special committee 
considering the merger of educational test- 
ing agencies, President McBride presented 
the committee’s recommendations to the 
full Board in April 1947, and since that time 
has participated in the creation of the new 
Educational Testing Service. Now as a 
member of the Board of Trustees of the 
new organization she will continue to have 
a large part in the advancement of educa- 
tional measurement. 














NEW LAW SCHOOL ADMISSION TEST 


The College Entrance Examination Board 
is cooperating with a number of law schools 
in the preparation and administration of an 
aptitude test designed to assist in deciding 
admissions. Participating in the project are 
the law schools of: 


Boston University 

Columbia University 

Cornell University 

Harvard University 

New York University 

Northwestern University 

Rutgers University 

Stanford University 

Syracuse University 

University of California 

(School of Jurisprudence ) 

University of Chicago 

University of Michigan 

University of Nebraska 

University of Pennsylvania 

Yale University 
The test, to be called the Law School Ad- 
mission Test, will be required for entrance 
in 1948 by a number of the law schools 
listed above. Since the roster of participat- 
ing law schools is at present not complete, 
candidates are advised to inquire from the 
institutions of their choice as to the desir- 
ability of registering for the test. 

The Law School Admission Test will be 
given on a nationwide scale on February 28 
and May 8, 1948. The Board will register 
candidates, establish centers for test admin- 
istration, score the papers, and report the 
scores to interested institutions and to the 
candidates. The fee is $10.00, payable by 
the candidate. Any surplus of receipts over 
expenditures will go first toward repaying 
developmental costs contributed jointly by 
the cooperating schools and the Board, and 
second toward the establishment of a fund 
for research. 

After the merger of educational testing 
organizations (see page 36), the College 
Board will continue direction of only its 
own college-entrance examinations. The 
Law School Admission Test, among others 
not for college entrance, will be taken over 
by the Educational Testing Service. 


PARTIAL INCREASE IN FEES 


Because of rising costs, it has been neces- 
sary for the Board to increase certain of its 
fees to candidates. The fee for a morning 
program of tests taken alone (Program 1, 2, 
or 3) remains unchanged, at $5.00; the fee 
for the Special Aptitude Test for Veterans 
taken alone (Program 5) also remains un- 
changed, at $6.00. The fee for Program 4 
(the afternoon achievement tests ) has been 
increased from $6.00 to $7.00. The fee for 
all candidates taking a “complete series” 
(both a morning and an afternoon series of 
tests) has been increased from $9.00 to 
$10.00. Expressed in tabular form, the new 
schedule of fees for the college-entrance 
tests is as follows: 

Program 1 taken alone (verbal and mathematical 
sections of the Scholastic Aptitude Test) $5.00 
Program 2 taken alone (verbal section of the SAT, 
and Intermediate Mathematics Test).......... $5.00 
Program 3 taken alone (verbal section of the SAT, 
and Comprehensive Mathematics Test)......$5.00 
Program 4 taken alone (one, two, or three after- 
noon tests, other than the Special Aptitude Test 

See VOID i555 ndcncicheudsrsccnttcteenie $7.00 
Program 5 taken alone (Special Aptitude Test for 

WOUND a cckceinsnsstecss didies ts caanéuadengse <ejetahainls a 
“Complete series” (both a morning and an after- 

noon program of tests) ..............cceeees $10.00 

The schedule of fees noted above takes 
effect with the April, 1948 series of college 
entrance tests. 


December Test Series Discontinued 


At its meeting in October, 1947, the Col- 
lege Entrance Examination Board decided 
to replace the December series of a - 
entrance tests with a January series. This 
change takes effect after the December 
series of 1947. (For a schedule of test dates 
in 1948, see the back copy of this issue of 
the Review.) The Board’s decision was 
made only after a questionnaire revealed 
that the dropping of the December series 
would inconvenience very few institutions. 

During the academic year of 1947-1948, 
five series of college-entrance tests were 
offered (in August, December, January, 
April, and June). The elimination of the 
December series will substantially reduce 
the Board’s administrative costs. 
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TESTING OFFICES OF THREE ORGANIZATIONS MERGE 
Educational Testing Service Formed 


College Board to Continue Control of Entrance Examinations 
By Henry CHAUNCEY 


Director, College Entrance Examination Board and President, Educational Testing Service 


After a year of careful consideration, the 
College Entrance Examination Board at its 
October 1947 meeting voted to merge its 
testing offices with those of other organiza- 
tions to form the Educational Testing Serv- 
ice (ETS). The ETS unites in a single 
organization the testing functions hitherto 
performed by the Carnegie Foundation for 
the Advancement of Teaching (through the 
Graduate Record Office), the College En- 
trance Examination Board, and the Amer- 
ican Council on Education (chiefly through 
the Cooperative Test Service, the Psycho- 
logical Examination, and the National Com- 
mittee on Teacher Examinations). On 
December 19, 1947, the new organization 
received its charter as a non-profit corpor- 
ation under the Education Law of the State 
of New York. 


The ex-officio members of the Board of 
Trustees of the Educational Testing Service 
are: 


Outver C. CARMICHAEL, President of the Car- 
negie Foundation for the Advancement of 
Teaching 

Epwarp S. Noyes, Chairman of the College 
Entrance Examination Board 

GerorcE F. Zoox, President of the American 
Council on Education 


The nine appointed trustees are: 


RayMOND ALLEN, President of the State Uni- 
versity of Washington 

Josern W. Banker, President of the Research 
Corporation, New York City, and former 
Dean of the Faculty of Engineering, Co- 
lumbia University 

James B. Conant, President of Harvard Uni- 
versity 

J. Wimu1aM Fursricut, U.S. Senator from 
Arkansas, and former President of the Uni- 
versity of Arkansas 

Herotp Hunt, Superintendent of Schools, 
Chicago, Ill. 
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KaTuaringE E. McBrive, President of Bryn 
Mawr College 

TxHomas R. McConne tt, Dean of the College 
of Science, Literature, and Arts, University 
of Minnesota 

Lester NELson, Principal of Scarsdale High 
School, Scarsdale, N.Y. 

Francis T. SPAULDING, Commissioner of Edu- 
cation, State of New York 


The Board of Trustees represents a broad 
range of interests and background. Amer- 
ican education will profit from their first- 
hand knowledge of the problems of the 
high school, the city school system, the state 
educational system, the educational services 
of the armed forces, public and private col- 
— and universities, and professional 
schools in law, medicine, engineering, and 
education. 

At the December 1947 meeting, the 
Board of Trustees elected Dr. Conant as 
their Chairman. Dr. Barker was chosen as 
Chairman of the Executive Committee, Dr. 
McConnell as Chairman of the Committee 
on Research, and Dr. McBride as Chairman 
of the Committee on Tests and Measure- 
ments. 


CHANGES TO BE GRADUAL 


The Educational Testing Service began 
formal operation on January 1, 1948, but 
for the immediate future each of its con- 
stituent offices will continue to operate as 
it has in the past. No sudden changes are 
to be expected in programs or procedures. 
The ETS will, for the time being, retain 
the staffs of the merging organizations, and 
will perform the present testing functions 
much as they are now being performed. 
The main offices of the ETS are in New 
York and Princeton, but eventually there 
will be branch offices in several sections of 
the country. 








While each merging group will for the 
present continue to carry on its own test- 
ing functions under the auspices of the ETS, 
plans are already being laid for the even- 
tual centralization and integration of the 
activities of the three groups. After that is 
done, the ETS should be in a position to 
make important contributions in its own 
right to American education, for the com- 
bined resources of the three groups, to- 
gether with a generous grant from the 
Carnegie Corporation, will make the ETS 
a far stronger organization than any of the 
testing offices from which it was formed. 
In addition to its regular testing activities, 
the ETS will be in a position to develop 
tests of high standards in a variety of fields 
as yet unexplored; it will be able to coordi- 
nate tests and testing programs at different 
educational levels; it will be able to provide 
effective advisory services to schools and 
colleges; and it will conduct and sponsor 
varied and extensive research projects. 


IMPLICATIONS FOR THE COLLEGE BOARD 


There exists, and quite understandably, 
some confusion as to precisely what sig- 
nificance the merger holds for the College 
Entrance Examination Board. Very simply, 
the College Board will turn over its Prince- 
ton operating office to the ETS and con- 
tract with the ETS to carry on the regular 
Board testing activities. The College Board 
will, however, continue as an educational 
association and will continue to hold its 
semi-annual meetings of representatives of 
member colleges and secondary school 
associations. 

Furthermore, while relinquishing control 
of its non-college-entrance testing activities, 
the College Board will retain complete 
control of its college-entrance examination 
program. Committees of examiners in the 
various subjects will continue to be ap- 
pointed by the College Board, and will have 
responsibility for preparation of the tests, 
with the advice and assistance of the tech- 
nical staff of the ETS. Thus, the Board will 
relinquish none of its control over the col- 
lege-entrance examination program, which 
will be, as it always has been, a direct re- 


flection of the needs and wishes of its mem- 
ber schools and colleges. THe CoLLEcE 
Boarp REviEw will continue publication on 
its regular schedule. 


BOARD TO CONCENTRATE ON COLLEGE 
ENTRANCE 


The new plan of organization makes it 
possible for the College Board to make full 
use of the services of a competent research 
and technical staff without being itself re- 
sponsible for the management of an oper- 
ating office. The College Board will also be 
relieved of the responsibility for adminis- 
tering the testing activities recently under- 
taken at the request of the government and 
other groups—activities not directly the 
concern of an institution devoted primarily 
to the examination of students for admis- 
sion to college. Thus, by transferring to the 
ETS complete jurisdiction for outside test- 
ing programs and calling upon it for tech- 
nical supervision of the college-entrance 
testing program, the College Board will free 
itself from “big business” and make possible 
an early return to the function for which it 
was established—the consideration of edu- 
cational issues relating to the selection of 
students for entrance into college. 


ADVANTAGES OF NEW AGENCY 


In brief, then, the Educational Testing 
Service will in no way interfere with the 
non-testing activities of the organizations 
which have brought it into being. Its func- 
tion will in general be confined to testing 
and measurement activities at the opera- 
tional and technical level, and it will con- 
stantly seek advice from expert sources out- 
side its own organization. Yet it will be in 
an excellent position to coordinate testing 
activities throughout the country and to 
conduct test research on a broad front. On 
the other hand, the College Board will be 
free to resume its deliberate role in the 
solving of educational problems of college 
admissions. 

These many advantages seem highly de- 
sirable to the College Board, and, with the 
other merging organizations, it looks for- 
ward with hope and confidence to the oper- 
ation of the new national testing agency. 
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THE PRELIMINARY ACTUARIAL EXAMINATIONS 


As a culmination of plans first laid in the 
spring of 1943, the College Entrance Ex- 
amination Board administered the Prelimi- 
nary Actuarial Examinations in June, 1947. 
Almost 900 candidates were tested at 68 
centers in the United States and Canada. 
“Cutting scores” to separate passes from 
failures on each of the separate examina- 
tions were set by the Joint Education and 
Examination Committee of the Actuarial 
Society of America and the American Insti- 
tute of Actuaries. Scores were reported to 
students as well as to the two actuarial 
organizations. 

The Preliminary Actuarial Examinations 
form the first three parts of an eight-part 
battery of examinations in the actuarial 
field. Examination 1, Language Aptitude, 
is a verbal aptitude test; a French form of 
this examination is optionally available for 
candidates taking the test at Canadian cen- 
ters. Examination 2, General Mathematics, 
covers college algebra, plane trigonometry, 
coordinate geometry, and calculus. Exami- 
nation 3, Special Mathematics, covers finite 
differences, probability, and statistics. The 
passing of Examinations 1-3 is prerequisite 


to the subsequent examinations which must 
be taken to qualify as an Associate or Fel- 
low of the Actuarial Society or of the Amer- 
ican Institute of Actuaries. 

Examination dates for the 1948 Prelimi- 
nary Actuarial Examinations are as follows: 

Examination 3: Friday afternoon, May 14 

Examination 1: Saturday morning, May 

15 
Examination 2: Saturday afternoon, May 
15 
The fee is $4.00 for one examination, $8.00 
for two, and $12.00 for all three. 

The Preliminary Actuarial Examinations 
administered in 1947 were prepared by a 
committee of examiners, including actuaries 
and college professors of mathematics, in 
cooperation with the staff of the Board. The 
work of this committee and of the Board 
staff has been guided by joint committees 
or subcommittees of the two actuarial or- 
ganizations, and has been financially sup- 
ported by these two organizations. After the 
merger (see page 36), the Preliminary Ac- 
tuarial Examinations will be taken over by 
the Educational Testing Service. 


Accuracy of Scoring the Objective Examinations 
(Continued from page 33) 


than any other), a check of over 1000 
papers revealed no errors of more than two 
raw-score points. (See Table 1, p. 43.) 

All these results were obtained on papers 
from the April 1947 test-series—the largest 
in the history of the Board—for the scoring 
of which it was necessary to recruit an un- 
usually large proportion of inexperienced 
scorers under relaxed hiring standards. 
With a return to more normal manpower 
conditions, the accuracy of scoring the Col- 
lege Board tests should become still higher. 


PLANNING FOR SCORING 


The planning for scoring must start 
months before any tests. are administered. 
Extrapolating from previous experience, a 
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prediction is made of how many candidates 
will take each test; and the necessary per- 
sonnel, space, ee. and schedule of 
operations are planned accordingly. In ad- 
dition, supervisory personnel are trained 
in the procedures to be followed when the 
test booklets and answer sheets finally ar- 
rive. In large degree, the success of scoring 
depends on the adequacy of these advance 


plans and preparations. 


STENCILS FOR SCORING 


Accurate stencils, properly labelled, are 
basic to any large-scale scoring operation. 
The original scoring-key for each test is 
prepared independently by at least two, 
and usually three, different persons. After 














PACKETING THE ANSWER SHEETS. Upon receipt, the answer sheets for each test are numbered 
serially and packeted into sets of 25 each. Each set is kept intact throughout the process of 
scoring and checking. 


a careful check for agreement, one of the 
keys is clipped i ccaale on a sheet of semi- 
transparent, greenish, plastic material (lu- 
marith) cut to the exact size and shape of 
the standard answer sheet for the test. A 
hole is punched through each marked an- 
swer of the key—and at the same time 
through the lumarith sheet underneath. The 
result is a stencil which is an exact copy 
of the original, checked key. A label, care- 
fully printed with India ink on a piece of 
adhesive tape, is then pasted on the stencil. 
As a guarantee against the use of a wrong 
stencil, one or more holes, corresponding 
to “landmarks” on the answer sheet for 
the test, are punched through the stencil. 
When the stencil is applied to the answer 
sheet, the landmarks are clearly visible 
through the holes. Because numerous copies 
of each stencil are generally required, exact 
duplicates of the first lumarith stencil are 
made by a power-punch. Labels are then 
added, and the whole is checked by “sight- 
ing” through the top (original) stencil. 
The process described above refers par- 


ticularly to the stencils employed in hand- 
scoring. The stencils for use in the electrical 
Test Scoring Machine are similarly pre- 
pared, except that a standard paper form 
from the International Business Machines 
Corporation proves more convenient than 
lumarith sheeting. 


ASSURANCE AGAINST LOSS OF ANSWER SHEETS 


Immediately following administration of 
a test, all examination materials are sent by 
first-class mail or express to Box 592, Prince- 
ton, N.J. The Receiving Section of the Col- 
lege Board sorts the materials, numbers 
each answer sheet serially, and packets the 
answer sheets into sets of 25 each. The 
count within each packet is then verified by 
the Scoring Section. Throughout the entire 
scoring and checking process, each packet is 
kept intact. Packeting the answer sheets not 
only facilitates the keeping of records of 
accuracy and production, but also aids in 
preventing the misplacement or loss of any 
answer sheet. 
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SCORING BY HAND 


All answer sheets are first hand-scored 
with the aid of the lumarith stencils already 
described, and then checked either by hand 
or by electrical test scoring machines. One 
important duty of every hand-scorer is to 
scan each answer sheet for multiple answers 
(two or more responses to a multiple-choice 
item for which only one response is al- 
lowed). All multiple responses which are 
found are thoroughly over-marked with a 
red wax pencil: the red warns the hand- 
scorer not to credit such responses, while 
the wax insensitizes the electrical scoring 
machine. In the case of answer sheets to be 
checked by machine, the hand-scorer also 
checks for frayed or torn answer sheets, use 
of the wrong type of pencil, incomplete 
erasures, stray dots left by a pencil resting 
on the answer sheet, and merely gratuitous 
scribbling—all of which may introduce er- 
rors in machine-checking, unless taken care 
of. 

In recent years, some organizations have 
dispensed with hand-scoring entirely, using 


CLEAROSE 
Hanp Scorinc. All answer sheets are first hand-scored, and then checked either by hand or 


instead the electrical test scoring machine 
of the International Business Machines Cor- 
poration. The College Board makes use of 
scoring machines only in the process of 
checking or re-scoring (see below). 


CHECKING 


After hand-scoring, every single answer 
sheet is checked either by hand or by the 
electrical test-scoring machine. The elec- 
trical scoring machine works through a 
series of brass “fingers” which make elec- 
trical contact with the marks on the paper 
(it is for this reason that a special “electro- 
graphic” pencil must be used in marking 
the answer sheet). Hand-scoring and ma- 
chine-checking complement each other 
nicely, each protecting against errors to 
which the other is occasionally subject. In 
the 1947 series, however, not all tests were 
adapted to machine-checking. 

Both in hand- and machine-checking, use 
is made of a “first checker” and a “second 
checker.” In _ hand-checking, the first 
checker verifies that the number of right, 





by machine. 
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CLEAROSE 
Macuine Cueckinc. The 1.B.M. Test Scoring Machine is used to check hand-scoring. 


wrong, and omitted responses equals the 
total number of items in the test. If any dis- 
crepancy is found, a correction is made, the 
paper is placed criss-cross in the packet, and 
the correction is verified by the second 
checker. An essentially similar process is 
followed in machine-checking. 

As a final step in the process of checking, 
all papers are scanned (a) for pages or tests 
which may not have been scored, and (b) 
for multiple answers which may have es- 
caped over-marking with red wax pencil. In 
addition, from each completed packet one 
paper is selected at random, and scored 
anew by a well-qualified supervisor. If any 
error is disclosed, the entire packet of 
papers is re-scored by hand. 


SELECTION OF PERSONNEL 


A constant check is kept both on the 
speed and accuracy of each scorer. Scorers 
with a low record of accuracy are promptly 
removed from scoring (manpower permit- 
ting). Scorers with a high record of accu- 


racy are among the first to be reassigned to 
scoring in the next scoring period. 

All checks on scoring are performed by 
highly experienced personnel. 


ADDING THE SCORES 


Most of the current tests of the Board re- 
ee the use of both sides of an answer 
sheet. Since each side of the answer sheet 
is scored separately, obtaining the total 
score for a test requires the adding of the 
part-scores. This adding is accomplished 
mentally by specially qualified personnel; 
and is checked by a second, experienced 
operator, making use of an adding machine. 


PUNCHING, VERIFYING, CONVERTING, 
SORTING, COLLATING, AND REPORTING 


After all scores have been checked and 
added, the answer sheets go to the “ma- 
chine room.” Here a variety of mechanical 
or quasi-mechanical operations lead first to 
a collection of data for each individual on 
all the tests he has taken, and finally to a 
mechanically printed report to the colleges 
of the individual's choice. 
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CLEAROSE 


SECTION OF THE “MacuinE Room.” Here the final steps of punching, verifying, converting, 
sorting, collating, and tabulating or printing of candidates scores are completed with the 
help of electrical punched-card equipment. 


The first step in the machine-room processes is 
the punching of a series of 1.B.M. tabulating-ma- 
chine cards tor each individual. The “score detail 
card” contains the individual’s examination-num- 
ber, his test score, and a number identifying the 
name of the test (a separate card is punched for 
each test taken by the individual). In addition, a 
“name card” is prepared, bearing both the indi- 
vidual’s examination-number and his name. A sec- 
ond person re-punches these cards on a “verifying” 
machine; in case of a discrepancy between the 
original punch and the verifying punch, a fixed 
routine is employed to correct the error. Next, the 
raw test scores for each test are mechanically con- 
verted to scores on the standard scale—which runs 
from 200 to 800, with 500 representing the score of 
an “average” or “normal” individual. (The equa- 
tions governing the transformation from raw to 
converted or standard scores are prepared by the 
Computing Section of the Statistical Analysis De- 
partment.) With completion of the conversion, the 
basic data for each individual are complete. The 
verified cards containing the converted scores are 
now sorted (on a “sorting machine”) according to 
examination-number. By means of a “collator” 
cards with the same examination-number are 
brought together; thus, if the individual with ex- 
amination no. 10713 took three tests, the three 
cards for this individual, plus the name-card, would 
be brought together immediately behind the cards 
for the individual with examination no. 10712. 
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After collating, the cards are returned to the sort- 
ing machine for classification according to college 
to which the report of test scores is to be issued. 
The cards are now ready for the “tabulating ma- 
chine.” The tabulating machine prints the individ- 
ual’s name, his examination-number, the names of 
the tests taken and the score on each test. This 
printed, tabulating-machine record is the report 
which is sent to the colleges. The accuracy of the 
tabulating machine is ordinarily considered perfect. 


APRIL 1947 RESULTS 


The various procedures described above 
have been designed to yield the highest ac- 
curacy possible, within reasonable demands 
of economy and speed. Accuracy, however, 
is not to be assumed, but verified. After 
every large test series, therefore, a random 
sample of answer sheets is selected and 
carefully scored again, by hand. (This scor- 
ing is performed by experienced personnel, 
under different supervision from the orig- 
inal scoring crew.) In April 1947, for ex- 
ample, a total of 6,768 test scores were 
re-checked. The results are shown in Table 
1. It is noteworthy that in the Scholastic 
Aptitude Test—which had been checked by 








machine—only small errors occurred; the 
only comparatively large error (of —10) 
occurred on one of the hand-checked tests. 
This is exactly what one would expect, from 
the nature of the hand- vs. the machine- 
checking processes. Since several small er- 
rors are less serious than a single large one, 
machine-checking appears preferable from 
the point of view of accuracy—and is cer- 
tainly preferable from the point of view of 
speed and economy. It is planned that vir- 
tually all the objective tests of the 1948 
series will be cast in forms which permit 
checking by machine. 

The errors listed in Table 1 are expressed 
in terms of raw-score units; the table does 
not, for example, show what a raw-score 
error of 1 amounts to in terms of the con- 
verted scores or “standard ratings” reported 
to the colleges. This varies somewhat from 
test to test: thus, in the verbal section of 
the SAT, containing 170 items, an error of 


TABLE 1 
Accuracy of Scoring, April 1947 







No. or PAPERS WITH 


SAT*—Verbale 
SAT*—Math-< . 
CMT >—Verbale 
CMT>—Math.¢ 
Social Studies . 
French Read. . 
German Read. 
Latin Read. . 
Spanish Read. 
Biology .... 
Chemistry ....... 
Physics ............ 
Spatial Rel. 


TOTAL ..........17, 


® Scholastic Aptitude Test, 1947. 

b> Comprehensive Mathematics Test, 1947. 

¢ Checked by I.B.M. Test Scoring Machine. 
Other tests were hand-checked. 

4 Includes one error of —4 and one of —6. 

e Error of —10. 




















1 raw-score point means less than in the 
mathematical section, containing 60 items. 
In general, an error of 1 raw-score point 
equals an error, on the standard scale, of 
from 4 to 10 points. Such an error, in com- 


parison with the range of scores on the 
standard scale (from about 200 to 800), is 
generally negligible. 


SUMMARY 


One of the largest and most responsible 
tasks of the College Entrance Examination 
Board is the yearly scoring of over 200,000 
test papers from more than 60,000 appli- 
cants to colleges. Despite the opposing de- 
mands of speed and economy, this task is 
executed with satisfactory accuracy. Thus, 
of about 8,000 scores submitted to careful 
re-check, only 3 scores were in error by two 
raw-score points, and only 3 were in error 
by more than two raw-score points. In the 
case of the Scholastic Aptitude Test, a check 
of over 1,000 papers revealed no error of 
more than 2 raw-score points. These results 
were obtained in a random sample of an- 
swer sheets from the April, 1947 testing. 

A brief description is given of the pro- 
cedures by which the accuracy of scoring 
is maintained. These procedures start with 
an estimate, several months in advance, of 
the personnel, space, and equipment that 
will be needed. Subsequent steps include: 
training of supervisory personnel; prepara- 
tion and checking of stencils for scoring; 
packeting or inventorying of answer sheets, 
as a precaution against misplacement or 
loss; scoring each side of the answer sheet, 
and checking the scores; adding of part- 
scores; and the final I.B.M. machine opera- 
tions of punching, verifying, converting, 
sorting, collating, and reporting. 

Probably most vulnerable, from the point 
of view of accuracy, is the process of scor- 
ing. Because hand-scoring and machine- 
checking complement each other nicely— 
each protecting against errors to which the 
other is subject—virtually all the objective 
tests for the 1948 series have been cast in 
forms that permit checking by machine. 
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DATES OF COLLEGE-ENTRANCE EXAMINATIONS 
The College Entrance Examination Board has set the following schedule | 
for its series of entrance examinations: 


Saturday, April 3, 1948 
Saturday, June 5, 1948 
Wednesday, August 18, 1948 
Saturday, January 15, 1949 
Saturday, April 9, 1949 


The Board’s BuLLETIN oF INFORMATION, obtainable on request, gives 

| detailed information concerning the examination-schedule, applications, | 

fees, examination centers, and the general nature of the examinations. 

| Attention is called particularly to the fact that candidates planning to take | 

| the Scholastic Aptitude Test must specify, in their application blank, 
whether this test is to be taken in Program 1, Program 2, or Program 3. | 
(Consult the BULLETIN oF INFORMATION. ) 

Applicants are urgently advised to obtain and complete their application | 

forms early. The normal closing dates for the receipt of applications in 

| Princeton, N.J. are as follows: for candidates east of the Mississippi River 

| or on either bank of the Mississippi, three weeks before the date of the ex- 
amination; for candidates west of the Mississippi River or in Canada, Mex- 
ico, or the West Indies, four weeks before the date of the examination; and 
for candidates outside of the United States, Canada, Mexico, and the West 
Indies, seven weeks before the date of the examination. A penalty fee of 
three dollars (which should accompany the application ) will be charged for 

| applications received later than the normal closing date. No applications 
received in Princeton later than one week before the examination date will 

| be considered. 





THE ENGLISH EXAMINATION FOR FOREIGN STUDENTS 


The English Examination for Foreign Students will be administered dur- 
ing the first week in March in foreign centers established by the Board. A 
list of the supervisors may be obtained from the Board. Colleges should in- 
form each candidate of the name of the supervisor nearest him; the super- 
_ visor will arrange the exact date of the administration in his center. Addi- 
tional or interim administrations can be arranged as often as once a month. 
| The English Examination for Foreign Students is also available for ad- | 
| ministration at any time by colleges in the United States. 
For details, write to the College Entrance Examination Board. 





